The Kolmogorov-Sinai entropy is a far reaching dynamical generalization of Shannon entropy of information systems. This entropy works perfectly for probability measure preserving (p.m.p.) transformations. However, it is not useful when there is no finite invariant measure. There are certain successful extensions of the notion of entropy to infinite measure spaces, or transformations with ...

The first part of this work considers the entropy of the sum of (possibly dependent and non-identically distributed) Bernoulli random variables. Upper bounds on the error that follows from an approximation of this entropy by the entropy of a Poisson random variable with the same mean are derived via the Chen-Stein method. The second part of this work derives new lower bounds on the total variat...

The first part of this work considers the entropy of the sum of (possibly dependent and non-identically distributed) Bernoulli random variables. Upper bounds on the error that follows from an approximation of this entropy by the entropy of a Poisson random variable with the same mean are derived via the Chen-Stein method. The second part of this work derives new lower bounds on the total variat...

Sufficient conditions are developed, under which the compound Poisson distribution has maximal entropy within a natural class of probability measures on the nonnegative integers. Recently, one of the authors [O. Johnson, Stoch. Proc. Appl., 2007] used a semigroup approach to show that the Poisson has maximal entropy among all ultra-log-concave distributions with fixed mean. We show via a non-tr...

An information-theoretic foundation for compound Poisson approximation limit theorems is presented, in analogy to the corresponding developments for the central limit theorem and for simple Poisson approximation. It is shown that the compound Poisson distributions satisfy a natural maximum entropy property within a natural class of distributions. Simple compound Poisson approximation bounds are...

We define the Boltzmann-Gibbs entropy of random finite set on a general space X as the integration of logarithm of density function over the space of finite sets of X , where the measure for this integration is the dominating measure over this space. We show that with a unit adjustment term, the same value for entropy can be obtained using the calculus of set integrals which applies integration...

We prove that the notions of Krengel entropy and Poisson entropy for infinite-measure-preserving transformations do not always coincide: We construct a conservative infinite-measure-preserving transformation with zero Krengel entropy (the induced transformation on a set of measure 1 is the Von Neumann–Kakutani odometer), but whose associated Poisson suspension has positive entropy. Résumé. Nous...